API: add top-level melt function as method #15521

ResidentMario · 2017-02-27T16:43:23Z

(progress on?) API: add top-level functions as method #12640
- pd.melt (DataFrame) (xref API: DataFrame.melt(), DataFrame.wide_to_long() #14876)
tests added / passed
passes git diff upstream/master | flake8 --diff
whatsnew entry

This is continued from PR#15513.

ResidentMario · 2017-02-27T16:44:27Z

How should this be tested?

jreback · 2017-02-27T16:51:15Z

where pd.melt is currently tests, you can add in a a test like

test_api_method or something and just copy an existing tests and subtitute DataFrame.melt for pd.melt (or changing a tests is ok too)

ResidentMario · 2017-02-27T16:57:41Z

Copying an entire test case would cause a lot of duplication though. Since DataFrame.melt is supposed to have the same method signature as pd.melt, can't we just replace melt with DataFrame.melt in the test bodies?

jreback · 2017-02-27T17:14:28Z

@ResidentMario yes you can simply replace them (though I would still add a single function to test the api (e.g. that pd.melt works)

jreback · 2017-02-27T17:49:10Z

also add this to api.rst. and in the reshaing.rst I would change the docs.

ResidentMario · 2017-02-27T18:21:03Z

And and addition to whatsnew, presumably. Where would that go under there?

jreback · 2017-02-27T18:24:22Z

you could put it in Enhancements

codecov-io · 2017-02-27T20:30:54Z

Codecov Report

Merging #15521 into master will increase coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master   #15521      +/-   ##
==========================================
+ Coverage   90.97%   90.97%   +<.01%     
==========================================
  Files         145      145              
  Lines       49483    49492       +9     
==========================================
+ Hits        45015    45024       +9     
  Misses       4468     4468

Flag	Coverage Δ
#multiple	`88.73% <100%> (ø)`	⬆️
#single	`40.63% <77.77%> (ø)`	⬆️

Impacted Files	Coverage Δ
pandas/core/frame.py	`97.57% <100%> (ø)`	⬆️
pandas/core/reshape.py	`99.28% <100%> (ø)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update da0523a...1657246. Read the comment docs.

ResidentMario · 2017-02-28T23:25:08Z

gtm?

jreback · 2017-03-02T13:31:43Z

doc/source/whatsnew/v0.20.0.txt

@@ -160,6 +160,7 @@ Other enhancements
 - ``Series.sort_index`` accepts parameters ``kind`` and ``na_position`` (:issue:`13589`, :issue:`14444`)

 - ``DataFrame`` has gained a ``nunique()`` method to count the distinct values over an axis (:issue:`14336`).
+- ``DataFrame`` has gained a ``melt()`` method for unpivoting from a wide to long format (:issue:`12640`).


note that this is the same as pd.melt

jreback · 2017-03-02T13:33:03Z

lgtm. @jorisvandenbossche

jorisvandenbossche · 2017-03-02T23:35:09Z

Will take a look tomorrow!

ResidentMario · 2017-03-12T22:13:32Z

@jorisvandenbossche bump :)

jreback · 2017-04-03T15:16:48Z

can you rebase, this looks good though (mainly to fix the whatsnew)

ResidentMario · 2017-04-04T00:58:58Z

Rebased.

jorisvandenbossche

Few doc comments

jorisvandenbossche · 2017-04-04T11:46:51Z

pandas/core/frame.py

@@ -105,7 +105,9 @@
    optional_by="""
        by : str or list of str
            Name or list of names which refer to the axis items.""",
-    versionadded_to_excel='')
+    versionadded_to_excel='',
+    versionadded_melt='\n.. versionadded:: 0.20.0\n',


this needs extra spaces (you can check pd.DataFrame.melt? if you are on this branch)

I'm not sure what you mean by this, can you clarify?

If you do pd.DataFrame.melt? you see:

Signature: pd.DataFrame.melt(self, id_vars=None, value_vars=None, var_name=None, value_name='value', col_level=None) Docstring: "Unpivots" a DataFrame from wide format to long format, optionally leaving identifier variables set. This function is useful to massage a DataFrame into a format where one or more columns are identifier variables (`id_vars`), while all other columns, considered measured variables (`value_vars`), are "unpivoted" to the row axis, leaving just two non-identifier columns, 'variable' and 'value'. .. versionadded:: 0.20.0 Parameters ---------- ....

so the versionadded is not indented as the other lines.

But I think Jeff fixed it before merging

jorisvandenbossche · 2017-04-04T11:46:58Z

doc/source/reshaping.rst

@@ -265,7 +265,7 @@ the right thing:
 Reshaping by Melt
 -----------------

-The :func:`~pandas.melt` function is useful to massage a


Can you mention here both?

Done (GitHub is not folding this review component for some reason, however).

jorisvandenbossche · 2017-04-04T11:49:33Z

pandas/core/frame.py

+    columns, considered measured variables (`value_vars`), are "unpivoted" to
+    the row axis, leaving just two non-identifier columns, 'variable' and
+    'value'.
+


This blank line can be removed

jorisvandenbossche · 2017-04-04T12:00:50Z

@ResidentMario Sorry my 'tomorrow' took a lot longer .. !

To be clear: I am in the meantime fully convinced of the usefulness of melt, and having it as method on a DataFrame as counterpart of the pivot method is really nice.

My main concern I have (or maybe not really concern, bur rather opportunity) is about the exact API. There have been raised some issues about this (#10109, #14876 (comment)), so I only think we should take this as an opportunity to look at the current melt API and see if we would implement it in the same way now.
And conclusion can well be that we just want to keep it as is.

For example, IMO it would be useful to make a comparison between the R melt and gather, see how Hadley has improved it in the new gather version and see if we can learn something of it.

If we don't find the time for this before 0.20 release, I am OK with merging this.

jreback · 2017-04-04T12:32:11Z

ok @ResidentMario pls make those corrections and we can merge. If after this (in next week or two) we can figure out a better API we can discuss, otherwise can do it in next major version.

Note that also like to incorporate (and deprecate) wide_to_long here.

btw should also look at cast and reshape R functions for inspiration.

jorisvandenbossche · 2017-04-04T13:44:59Z

-1 on incoroporating wide_to_long. As far as I know, it is hardly used, and has an API that is not really 'pandonic'.

ResidentMario · 2017-04-04T15:20:54Z

stack, unstack, melt, etcetera...agree that studying the possible advantages of data transforms in other languages would be a good thing.

I don't see this being done in the next two weeks however.

jreback · 2017-04-04T15:49:26Z

@ResidentMario

stack, unstack, melt, etcetera...agree that studying the possible advantages of data transforms in other languages would be a good thing.

I don't see this being done in the next two weeks however.

not a problem

jreback · 2017-04-04T17:53:00Z

thanks @ResidentMario

I made the doc-string a tiny bit more flexible (so the examples are more clear), but great patch!

jreback added API Design Reshaping Concat, Merge/Join, Stack/Unstack, Explode labels Feb 27, 2017

ResidentMario mentioned this pull request Feb 27, 2017

API: DataFrame.melt(), DataFrame.wide_to_long() #14876

Closed

jreback reviewed Mar 2, 2017

View reviewed changes

jreback added this to the 0.20.0 milestone Apr 3, 2017

ResidentMario added 6 commits April 3, 2017 20:54

+pd.DataFrame.melt.

d54dc2f

Melt docstring.

3cbbed5

rm stray debug.

11f3fe4

Add tests, update docs.

ff895fe

+whatsnew

5f306a9

tweak whatsnew entry.

28a38f2

ResidentMario force-pushed the 12640 branch from 9c2fe24 to 28a38f2 Compare April 4, 2017 00:57

jorisvandenbossche reviewed Apr 4, 2017

View reviewed changes

two doc changes

1657246

jreback closed this in e50d397 Apr 4, 2017

jreback mentioned this pull request Apr 4, 2017

API: add top-level functions as method #12640

Closed

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

API: add top-level melt function as method #15521

API: add top-level melt function as method #15521

ResidentMario commented Feb 27, 2017 •

edited

Loading

ResidentMario commented Feb 27, 2017

jreback commented Feb 27, 2017

ResidentMario commented Feb 27, 2017

jreback commented Feb 27, 2017

jreback commented Feb 27, 2017

ResidentMario commented Feb 27, 2017

jreback commented Feb 27, 2017

codecov-io commented Feb 27, 2017 •

edited by codecov bot

Loading

ResidentMario commented Feb 28, 2017

jreback Mar 2, 2017

jreback commented Mar 2, 2017

jorisvandenbossche commented Mar 2, 2017

ResidentMario commented Mar 12, 2017

jreback commented Apr 3, 2017

ResidentMario commented Apr 4, 2017

jorisvandenbossche left a comment

jorisvandenbossche Apr 4, 2017

ResidentMario Apr 4, 2017

jorisvandenbossche Apr 5, 2017

jorisvandenbossche Apr 4, 2017

ResidentMario Apr 4, 2017

jorisvandenbossche Apr 4, 2017

jorisvandenbossche commented Apr 4, 2017

jreback commented Apr 4, 2017

jorisvandenbossche commented Apr 4, 2017

ResidentMario commented Apr 4, 2017 •

edited

Loading

jreback commented Apr 4, 2017

jreback commented Apr 4, 2017

API: add top-level melt function as method #15521

API: add top-level melt function as method #15521

Conversation

ResidentMario commented Feb 27, 2017 • edited Loading

ResidentMario commented Feb 27, 2017

jreback commented Feb 27, 2017

ResidentMario commented Feb 27, 2017

jreback commented Feb 27, 2017

jreback commented Feb 27, 2017

ResidentMario commented Feb 27, 2017

jreback commented Feb 27, 2017

codecov-io commented Feb 27, 2017 • edited by codecov bot Loading

Codecov Report

ResidentMario commented Feb 28, 2017

jreback Mar 2, 2017

Choose a reason for hiding this comment

jreback commented Mar 2, 2017

jorisvandenbossche commented Mar 2, 2017

ResidentMario commented Mar 12, 2017

jreback commented Apr 3, 2017

ResidentMario commented Apr 4, 2017

jorisvandenbossche left a comment

Choose a reason for hiding this comment

jorisvandenbossche Apr 4, 2017

Choose a reason for hiding this comment

ResidentMario Apr 4, 2017

Choose a reason for hiding this comment

jorisvandenbossche Apr 5, 2017

Choose a reason for hiding this comment

jorisvandenbossche Apr 4, 2017

Choose a reason for hiding this comment

ResidentMario Apr 4, 2017

Choose a reason for hiding this comment

jorisvandenbossche Apr 4, 2017

Choose a reason for hiding this comment

jorisvandenbossche commented Apr 4, 2017

jreback commented Apr 4, 2017

jorisvandenbossche commented Apr 4, 2017

ResidentMario commented Apr 4, 2017 • edited Loading

jreback commented Apr 4, 2017

jreback commented Apr 4, 2017

ResidentMario commented Feb 27, 2017 •

edited

Loading

codecov-io commented Feb 27, 2017 •

edited by codecov bot

Loading

ResidentMario commented Apr 4, 2017 •

edited

Loading